Introduction to Neural Networks "Energy" and attractor networks Hopfield networks

‡ Supervised learning. Introduced the idea of a “cost” function over weight space Regression and learning in linear neural networks. The cost was the sum of squared differences between the networks predictions of the correct answers and the correct answers. The motivation was to derive a “learning rule” that adjusts (synaptic) weights to minimize the discrepancy between predictions and answers. Last time we showed 4 different ways to find the generating parameters {a,b} = {2, 3} for data with the following generative process: rsurface@a_, b_D := N@Table@8x1 = 1 RandomReal@D, x2 = 1 RandomReal@D, a x1 + b x2 + 0.5 RandomReal@D 0.25<, 8120<D, 2D; data = rsurface@2, 3D; Outdata = data@@All, 3DD; Indata = data@@All, 1 ;; 2DD;